Papers with Llama models
Culture Cartography: Mapping the Landscape of Cultural Knowledge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can empower users to be more knowledgeable, productive, and creative, but their utility is often diminished for under-represented groups and cultures. |
| Approach: | They propose a methodology that operationalizes a mixed-initiative approach to finding culture-specific knowledge that is salient to in-group users but unknown to LLMs. |
| Outcome: | The proposed method improves the accuracy of LLMs on culturally-competent language models by 19.2%. |
Hallucination Detox: Sensitivity Dropout (SenD) for Large Language Model Training (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on identifying and addressing hallucinations in large language models (LLMs), but the impact of the training process on hallucinosity remains underexplored. |
| Approach: | They propose a training protocol to reduce hallucination variance by dropping embedding indices with significant variability and an unsupervised halluciation detection metric, Efficient EigenScore. |
| Outcome: | The proposed training protocol reduces hallucination variance during training by dropping embedding indices with significant variability. |
Speculative Decoding Speed-of-Light: Optimal Lower Bounds via Branching Random Walks (2026.eacl-long)
Copied to clipboard
| Challenge: | Speculative generation has emerged as a promising technique to accelerate inference in large language models (LLMs) however, the fundamental limits on the achievable speedup remain poorly understood. |
| Approach: | They propose to draw a parallel token generation process and branching random walks to achieve the first "tight" lower bounds on the runtime of any deterministic speculative generation algorithm. |
| Outcome: | The proposed method reduces inference latency without altering the output distribution. |
Argument-Based Consistency in Toxicity Explanations of LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to evaluate free-form toxicity explanations are overly relying on input text perturbations. |
| Approach: | They propose a multi-dimensional criterion to evaluate LLMs' reasoning about toxicity . they conduct experiments on three Llama models and an 8B Ministral model . |
| Outcome: | The proposed criterion measures the extent to which LLMs’ free-form toxicity explanations reflect an ideal and logical argumentation process. |
Detecting Subtle Biases: An Ethical Lens on Underexplored Areas in AI Language Models Biases (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly embedded in the daily lives of individuals across diverse social classes. |
| Approach: | They propose to analyze LLMs' responses to 1,016 scenarios categorized into ethical, unethical, and neutral types. |
| Outcome: | The proposed model analyzed 1,016 scenarios categorized into ethical, unethical, and neutral types. |
Why and How LLMs Benefit from Knowledge Introspection in Commonsense Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can improve commonsense reasoning by generating intermediate knowledge, but the effectiveness of this knowledge introspection is not always guaranteed. |
| Approach: | They propose a training-free strategy that optimizes introspection via two stages: Knowledge Detection and Knowledge Regeneration. |
| Outcome: | The proposed approach mitigates the limitations of standard introspection and has consistent performance gains across all settings. |
Assessing the State of the Art in Scene Segmentation (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in scene segmentation have made it difficult to detect scenes in literary texts. |
| Approach: | They propose to modify existing models to improve detection of scenes in literary texts . they propose to use a training sample generation scheme to alleviate this problem . |
| Outcome: | The proposed model is more robust to different types of texts, while its overall performance is slightly worse than that of BERT-based models. |
Automatically Generated Definitions and their utility for Modeling Word Meaning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Modern language models generate semantic representations for words based on context and context based models. |
| Approach: | They propose to use dictionary-like sense definitions to generate sentence embeddings . they evaluate the quality of the generated definitions on existing English benchmarks based on the results of their study . |
| Outcome: | The proposed model sets new state-of-the-art results on lexical semantics tasks compared to baselines . |
SAGE: Sign-Adaptive Gradient for Memory-Efficient LLM Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train LLMs consume memory equivalent to twice the model size, resulting in a hybrid design that reverts to AdamW and negates the memory gains. |
| Approach: | They propose a new, memory-efficient O(d) adaptive scale that replaces AdamW in a hybrid structure that combines a Lion-style update direction with a memory-saving adaptive scale. |
| Outcome: | The proposed model outperforms existing methods on LLMs up to 1.3B parameters while significantly reducing optimizer state memory. |
From Representational Harms to Quality-of-Service Harms: A Case Study on Llama 2 Safety Safeguards (2024.findings-acl)
Copied to clipboard
Khaoula Chehbouni, Megha Roshan, Emmanuel Ma, Futian Wei, Afaf Taik, Jackie Cheung, Golnoosh Farnadi
| Challenge: | Recent advances in large language models have also introduced additional safety risks and raised concerns regarding their detrimental impact on already marginalized populations. |
| Approach: | They propose to use LLMs to evaluate their safety responses on already mitigated biases by evaluating models on already encoded assumptions. |
| Outcome: | The proposed model can encode harmful assumptions, but it can also be harmful for certain demographic groups. |
Deputy: Accelerating Large Language Model Inference with Dynamic Low-Rank Substitution (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing dynamic schemes such as early-exit and layer-drop reduce FLOPs but break batch processing or introduce KV-cache inconsistency. |
| Approach: | They propose a dynamic low-rank substitution framework that employs a lightweight decision module at each layer to dynamically determine the execution branch for different tokens. |
| Outcome: | The proposed model reduces computation by approximately 40% compared to the original dense model while outperforming existing baseline methods. |
Vulnerability of LLMs’ Stated Belief? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly employed in question-answering tasks. |
| Approach: | They analyze how different persuasive strategies influence stated belief stability . they also examine whether verbalized confidence prompting increases vulnerability . |
| Outcome: | The proposed model exhibits extreme compliance, with 82.5% of belief changes occurring at the first persuasive turn. |
Reinforced Efficient Reasoning via Semantically Diverse Exploration (2026.acl-long)
Copied to clipboard
Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin, Xin Xin
| Challenge: | Existing methods for reinforcement learning with verifiable rewards suffer from limited exploration diversity and inefficient reasoning. |
| Approach: | They propose a method that rewards concise and correct reasoning while penalizing unnecessarily long reasoning chains. |
| Outcome: | Extensive experiments on Qwen and Llama models validate the effectiveness and efficiency of ROSE. |